_ Best Availabl^opy 

PCX ^ 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PA TENT COOPERATION TREATY (PCT) 

WO 98/06216 




(51) International Patent Classification 6 : 
H04N 1/32, 7/08 



Al 



(11) International Publication Number: 
(43) International Publication Date: 12 February 1998 (12.02.98) 



(21) International Application Number: PCT/GB97/02060 

(22) International Filing Date: 1 August 1997 (01.08.97) 



(30) Priority Data: 
96 1 657!. 7 



7 August 1996 (07.08.96) 



GB 



(71) Applicant (for all designated States except US): CENTRAL 

RESEARCH LABORATORIES LIMITED [GB/GBl; Daw- 
ley Road. Hayes. Middlesex UB3 IHH (GB). 

(72) Inventor; and 

(75) Inventor/Applicant (for US only): TODD. Martin, Peter 
(GB/GBJ; 17 Honeycroft Hill, Uxbridge. Middlesex UHIO 
9NQ (GB). 

(74) Agent: LEAMAN. Keith; QED Patents Limited, Dawley Road. 
Hayes. Middlesex UB3 IHH (GB). 



(81) Designated SUtes: GB, JP, US. 



Published 

With international search report. 



(54) Title: METHOD AND APPARATUS FOR CODING INFORMATION 



(57) Abstract 



A method for inserting coded information into an image comprising the steps of analysing the image, identifying the edge regions 
within the image, determining the strength of the edge region, and producing an edge function which is dependent on the strength of the 
edge. The edge function is added to the image in order to represent coded information. 



BNSOOCID- <WO_9806216A1J_> 




FOR THE PURPOSES OF INFORMATION ONLY 
Codes used to identify Slates party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 




l-esotho 


SI 


Slovenia 


AM 


Armenia 


FI 


Finland 


LT 


Lithuania 


SK 


Sk>vakia 


AT 


Austria 


FR 


Prance 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


GatKrn 


LV 


Latvia 


SZ 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Heracgovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 
Burkina Faso 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


IIU 


Hungary 


ML 


Mali 


TT 


Trinidad arxl Tobago 


BJ 


Bcain 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BK 
BY 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


UZ 


Uzbekistan 


CF 


General African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CC 


Congo 


KE 


Kenya 


NL 


Netherlands 


vu 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Nofway 


zw 


Zimbabwe 


CI 


C6te d*l voire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






cu 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






OF. 


Germany 


LI 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SC 


Singapore 








wo 98/06216 PCT/GB97/02060 

1 

METHOD AND APPARATUS FOR CODING INFORMATION 

The present invention relates to a method and apparatus for the insertion, and 
5 subsequent decoding, of coded information into images. 

It is known to insert codes into images for example video transmissions or video 
clips or stills transmitted across a telecommunication link, for the purpose of identifying 
the owner of the images. There are a number of known schemes for inserting 
identification codes into the sync periods, and more recently it has been proposed to 
10 insert identification codes into the image itself, but in such a manner that the code cannot 
be detected by the eye. 

WO 95/14289 discloses the embedding of an identification code throughout an 
image by modulating a digitised version of the image with a small noise signal. The 
sp>ecific system described suffers from the disadvantage of requiring to have to hand the 
15 original image for code identification. Further improvements in code robustness for 
transmission over telecommunications links or broadcasting are also desirable. 

WO 95/20291 discloses a method of hiding copyright related messages with a 
digital data work, which relies on commonly occurring patterns or sequences of data in 
the work acting as signposts to target data elements which are modified according to 
20 certain rules. The disclosed method suffers from a lack of robustness to signal 
degradation. 

The above references suffer from a disadvantage that they are concerned wholly 
or principally with the digital domain, and the disclosed techniques are not suited to the 
analog domain, in particular where digital to analog conversion and analog to digital 

25 conversion may easily lose or degrade individual pixel values. 

In our copending application PCT/GB96/00246 (our ref PQ12632), there is 
disclosed and claimed a method for inserting coded information into an image, 
comprising analysing the image, identifying strongly featured regions and inserting 
coded information into these regions. Because the method relies on an analysis of the 

30 entire image and code being inserted in strongly featured regions, rather than in 
individual pixels, as in the prior art, the code is better able to survive analog to digital 
conversions and digital to analog conversions, where there will inevitably be pixel 
misalignments between the original and processed images, and hence the pixel values 
will be apparently altered. 



BNSDOCID- <WQ 0aQ62ieA1 I > 




wo 98/06216 PCT/GB97/02060 

2 

By "strongly featured regions'' is meant regions of primary strength to which 
the eye responds in viewing an image, for example, textured regions or lines or 
boundaries between two regions of different luminance. In such regions, it is possible to 
insert a relatively large amount of information without significantly altering the image in 
5 its appearance to the eye. Edge regions between areas of different luminance are 
preferred since these are very distinctive and will permit code insertion without visibly 
degrading the image, and the coded information is inserted into such strongly featured 
regions by altering the structure of the image in such region in a predictable or 
identifiable manner. The image is altered by applying an insert function to an area 
10 including the strongly featured region, which insert function gradually decreases in 
intensity from the centre of its region of application, so as to blend with the surrounding 
region. 

A main advantage of applying the coded information by altering the image in a 
predictable or identifiable manner, is that the information can be recovered in a decoding 

1 5 process at a remote location without having the original to hand. 

As disclosed in PCT GB96/00246 the image, for example, a video frame or field, 
is divided up into a number MxN of blocks in M rows and N columns, each block 
comprising nxn pixel elements (e.g. 8x8). Strongly featured regions are searched in each 
block for insertion of code. In order to encode a significant amount of information into 

20 an image, it is necessary to apply the insertion function to a number of edges in the 
image. If for example, one edge is chosen in each block into which the image is divided, 
then in order to increase confidence when attempting to recognise the code, the edges in 
one row may be encoded according to two separate pseudo-random codes, representing a 
"r'or"0". 

25 Prior to inserting the information, the image is assessed as to whether the image is 

suitable for code insertion, for example, the degree of consistency of the edge, the 
definition of the edge centre to prevent insertions when the image is determined to be 
unsuitable. Problems have arisen in the method described in PCT/GB96/00246 where 
more than one distinct edge is present in the image, or the part of the image under 

30 consideration for code insertion; for example there may be a comer feature. 
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An object of the present invention is to provide a method of embedding coded 
information into images by adding into the image a function which is dependant on the 
edge activity in the image. 

The present invention accordingly provides in one aspect a method for inserting 
coded information into an image, comprising the steps of analysing the image, 
identifying edge regions calculating the strength of the detected edge and producing an 
edge function dependent on the strength of the edge, and adding the edge function to the 
image in order to represent coded information. 

In accordance with the present invention, an image (or block within an image) 
may be selected to be encoded using an edge detection based scheme, the selection being 
dependant on the edge activity in the block. This allows for encoding of blocks with 
multiple features such as blocks containing a comer or block containing two edges. In 
these blocks, an edge detection scheme is appHed to the block to produce a function 
within the block which has a value at each pixel, which is related to the strength of the 



edge at that pixel. This edge function is suitably scaled, and optionally multiplied by a 
circular function centred on the centre of the block. The scaled edge function is then 
added to the image on a pixel by pixel basis, with the sign of the function, plus or minus, 
representing a "1" or "0" bit of information. In a decoding operation, edges are detected 
and an estimate is made of the appropriate edge function. This estimate is then correlated 
with the image in order to reveal the existence of coded information. 

In another aspect, the invention provides a method of decoding information 
contained in an image, the method comprising analysing the image, identifying edge 
regions and providing an estimate of the edge function dependent on the strength of the 
edge, and correlating the estimated edge function with the image in order to identify 
coded information present within the image. 

As preferred, the image may be divided into MxN blocks, and an encoding/ 
decoding operation is carried out on each block. Further, in accordance with the 
invention, an assessment is made of the type of features present in the image or image 
block, and a coding function is added to the image, the type of function depending on the 
type of image, as more fully described in our Application PCT/GB96/00246. Thus if 
only a single main edge is present, then an elliptical function may be added centred on 
the edge, if no distinct feature is present, then a circular function may be added, and if 
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more than one distinct edge is present, then the edge function of the present invention 
may be added. 

Another aspect of the present invention concerns synchronising the decoding 
process. An encoded image may be presented to the decoder in a geometrically altered 
5 state from which it was encoded. The geometric transformation may be for example an 
offset (x,y) translation, or a scaling in one or both axes (including different scaling in 
each axis), or a rotation. If the geometric transformation is of a significant size, then in 
order to decode it, this geometric distortion must be accounted for. The decoder 
synchronising process achieves this by an analysis process involving splitting the image 
10 into blocks, analysing the blocks for the type of insert and decoding the block 
accordingly. The process produces a measure of coded insets and their centres and the 
regularity of their structure. The offsets, scaling and rotations are then calculated from 
the geometric variations of the centres and structure. This is then used to correct the 
image before the final decoding. 

15 

Brief Description of the Drawings 

A preferred embodiment of the invention will now be described with reference to 
the accompanying drawing wherein: 

Figures 1 to 3 are diagrams for explaining the preferred method of the present 
20 invention. 

Description of the Preferred Embodiment 

The encoding and decoding algorithms of the preferred embodiment are listed 
below as a sequence of steps. 



25 Encoding Algorithm 

1) split the frame into M*N adjacent blocks each of n*n pixels 

2) calculate the local orientation for each pixel in each block 

3) calculate the edge detection function for each block. 

4) calculate the dominant orientation for each block, the amount of activity in each 
30 block, and the consistency of the dominant orientation 

5) to encode one data bit process each block within a predefined set of blocks 
distributed throughout the image, as follows: 
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5a) look up the next element of the PRS (psuedo random sequence) for the 

data bit 

1) if it is a 1 set to add 

2) if it is a -1 set to subtract 

5 5b) segment the blocks in the following categories 

a) a single edge/line in a dominant orientation 

b) a low activity block 

c) several major lines or edges 
5c) process the blocks as follows :- 

10 a) add/subtract an elliptical function 

al ) centred on the edge 
a2) aligned to the block orientation 
a3) scaled by the activity in the block 

b) add/subtract a circular function 

15 b 1 ) centred on the centre of the block 

b2) fixed strength 

c) add/ subtract the edge detection function 

cl) centred on the centre of the block, and modulated by a 
circular function. 

20 c2) scaled to bring it within the magnitude range of the 

elliptical function in a). 
6) repeat step 5) until all the bits are encoded. 
Decoding Algorithm 

1) split the frame into M*N adjacent blocks each of n=^n pixels 
25 2) calculate the local orientation for each pixel in each block 

3) calculate the edge detection function for each block. 

4) calculate the dominant orientation for each block, the amount of activity in each 
block, and the consistency of the dominant orientation 

5) to decode one data bit process each block within the predefined set of blocks 
30 distributed throughout the image, as follows:- 

5a) segment the blocks in the following categories 

a) a single edge/line in a dominant orientation 

b) a low activity block 
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c) several major lines or edges 
5b) process the blocks as follows :- 

a) calculate the concave/convex-ness of an elliptical function 
a 1 ) centred on the edge 

5 a2) aligned to the block orientation 

a3) scaled by the activity in the block 

b) calculate the concave/con vex-ness of a circular function 
b 1 ) centred on the centre of the block 

b2) fixed strength 
10 c) calculate the concave/convex-ness of an edge detection 

function 

cl) centred on the centre of the block and modulated by a 
circular function 

c2) scaled to bring it within the magnitude range 
15 5c) correlate the convex/concave-ness of the block with the next element of 

the data bit 1 PRS and accumulate the result. 

5d) correlate the convex/concave-ness of the block with the next element of 
the data bit 0 PRS and accumulate the result. 

6) compare the data bit I accumulated correlation with the data bit 0 accumulated 
20 correlation. The larger of the two is the decoded data bit. The size of the correlation is 

the confidence in the result. 

7) repeat steps 5) and 6) for each data bit. 



25 Referring now to Figures 1 to 3, the frame axis conventions used to describe the 

mathematics are shown in Figure 1, A pixel is defined by its coordinates (x,y) and its 
luminance value is r(x,y). Note that the top left pixel of an image is the (0,0) pixeK 
and that the y axis has its positive direction down the frame. 

As shown in Figure 2, the frame is segmented into non-overlapping blocks, each 

30 block being of size n by n pixels. Smaller values of n mean that it is more likely that 

only a single edge will be found in any given block. It also means that more 
individual blocks will be available for the correlation process. Larger values of n 
mean that larger, and therefore more robust, in.sertions can be made. In practice a 
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good choice of n is 8. The block axis conventions used in segmenting the frame 
are shown in Figure 2. 

Referring to Figure 3, the local orientation for each point in the frame is 
calculated from four surrounding points by a process as described below. This gives a 
5 vector at each point in the frame, with the magnitude of the vector representing the 

strength of the feature, and the angle representing twice the local orientation. This is 
illustrated in Figure 3. Thus the local orientation gives a measure of gradient of 
luminance in a particular direction within the block. A very large value of orientation 
indicates the existence of an edge; In this double angle form, the vectors can be 
10 vector-averaged over a block, to give the local orientation for the block. This 

provides a relatively fast estimating algorithm, 
e.g., as shown in Figure 3: 

e — 45degrees : by convention, 0 is associated with point a = (x,y) 
-dx - 0.7 
-dy - -0.7 

G is estimated from 
a = r(x,y) 
b = r(x,y + 1) 
c = r(x + l,y) 
d = rCx + l,y + 1) 
15 e = d-a 

f = b-c 
Re = -2 * e * f 
Im = e* - f ■ 

2 KRe) 
0 is in single angle form 
Re,Im are in double angle form 

orientations are averaged in the Re,Im double angle form. 

20 In the present invention, the measurement of luminance gradient is employed not 

only to give an assessment of the type of image in the block, and for determining the 
location of edges within the block as disclosed in PCT/GB 96/00246, but also to 
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determine the value of the edge detection function which is to be added to the image 
information in order to insert coded information within the image. 

The edge detection function is calculated by taking the magnitudes of the 
5 orientation- vectors for each pixel, then for each pixel the average of the magnitudes of 
it's four neighbours is calculated. I.e.for pixels (a) to (t) arranged in 4 rows of 5 
columns, thus:- 



a 


b 


c d e 


the averages 


0 0 0 0 0 


f 


g 
1 


h i j 


are calculated 


0 (a-fb+f+g)/4 (b-K;+g+h)/4 (c+d+h+i)/4 


k 


m n o 


as shown 


0 (f-hg+k+l)/4 (g-hh-f-l+m)/4 ( h-Hi+mH-n)/4 


P 


q 


r s t 




0 (k+l-hp+q)/4 (l-f m+q+rV4 .etc. 



10 The average magnitude for each pixel is then multiplied by a circular function centred on 
the center of the block. 

The edge detection function has a single number per pixel in the block but with the outer 
pixels set to zero. This is to say, that in the example discussed above, the edge detection 
function for pixels (a) to (e), (f), (j), (k), (o) and (p) to (t) are set to zero. This gives a 
1 5 pattern of values for the edge function thus:- 



0 


0 


0 


0 


0 


0 


0 


0 


0 


0.0 


0.0 


0.0 


0.2 


0.0 


0.0 


0 


0 


0.1 


0.2 


0.0 


0.6 


0.1 


0.0 


0 


0 


0.1 


0.1 


0.0 


0.9 


0.0 


0.1 


0 


0 


0.1 


0.2 


0.0 


0.7 


0.0 


0.1 


0 


0 


0.1 


0.2 


0.0 


0.1 


0.0 


0.1 


0 


0 


0 


0 


0 


0 


0 


0 


0 



The edge detection function is scaled to get it in the range 0.0 to 1 .0, so that it is in the 
same range as an elliptical function calculated for blocks having just a single dominant 
20 edge as described in PCT/GB96/00246. This is done by taking the magnitude at each 
pixel, scaling it by a fixed factor, and limiting it between zero and a fixed factor. 

The scaled edge detection function is then added to the original image. 

SUBSTUUTE SHEET (RULE 26) 
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CLAIMS 

1 . A method for inserting coded information into an image, comprising the steps of 
5 analysing the image, identifying edge regions, determining the strength of the edge 

region and producing an edge function which is dependent on the strength of the edge, 
and adding the edge function to the image in order to represent coded information. 

2. A method of decoding information contained in an image, the method comprising 
10 analysing the image, identifying edge regions and providing an estimate of the edge 

function dependent on the strength of the edge, and combining the estimated edge 
function with the image in order to identify in a correlation operation coded information 
present within the image. 

15 3. A method as claimed in any preceding claim, comprising the step of dividing up 
the image into M*N blocks of pixels, and performing the aforesaid steps in each block of 
the image, 

4. A method according to any one of the preceding claims comprising the steps of 
20 dividing the image into a plurality of blocks of mxn pixels (where m and n are any whole 

number), calculating orientation vectors for each pixel in each block and calculating the 
average mangitude of the vector for each selected pixel by calculating the average of the 
vector magnitudes of a plurality of pixels immediately adjacent the selected pixel, scaling 
the average magnitude of the vector for each selected pixel to produce a scaled edge 
25 function, and adding the scaled edge function to the image to represent coded 
information. 

5. A method according to claim 4 wherein the average magnitude of the vector for 
each selected pixel is multiplied by a circular function centred on the centre of each 

30 block to produce an edge function, the edge function is scaled to produce a scaled edge 
function, and the scaled edge function is added to the image to represent coded 
information. 




wo 98/062 16 PCT/GB97/02060 

10 

6. A method of decoding information contained in an image, the method 
comprising analysing the image, identifying strongly featured regions, determining for at 
least one such region an anticipated insertion of coded information, and combining such 
anticipated insertion with the image to determine in a correlation operation whether there 
5 has been inserted into the strongly featured region coded information, wherein in order to 
take account of geometric transformation of the image to be decoded, an inital decoding 
operation is carried out to determine the periodicity and/or regularity of the anticipated 
insertions, whereby to determine any such geometric transformation. 



10 7. A method of decoding according to claim 6 comprising the steps of sphtting the 
image into blocks, analysing the blocks for the type of insert, decoding the blocks to 
produce a measure of coded inserts and their centres and the regularity of their structure, 
calculating offsets, scaling and rotation from the geometric variations of the centres and 
structure and correcting the image prior to decoding the embedded codes. 

15 

8. A method for inserting coded information into an image substantially as herein 
described with reference to the accompanying drawings. 

9. A method of decoding coded information contained in an image substantially as 
20 herein described with reference to the accompanying drawings. 
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